User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0

Steps to reproduce:

Was innocently reading an email
Compacting spontaneously began
(I've selected the option to automatically compact when savings are 100 MB)
(I doubt this makes a difference, but all mail accounts are POP3)

Actual results:

All messages in all folders no longer visible. The number of messages in messages are still displayed, but no matter where I go in the folder tree the message windows says "no messages found".
The messages are all there, but I can't see them.
Only remedy is shutdown/restart Tbird

Expected results:

During and after a folder compaction, folders should be fully functional and all messages should be shown.
This is a new bug somewhere during v128.x on Windows 10

I am experiencing this on my macOS 14.7.4 (23H420) since updating to 137.0b1

Seems to happen with auto compact.. as it just did this while i was watching - went to auto compact and then all the messages in all the folders disappeared, but the "numbers of" still had a count.

This seemed to only happens for 1 account, but maybe that is the one being compacted, however, checked and another account has them missing

A repair folder does not work, it does refresh the numbers and the size, but no msgs reappear. - Only a TB restart

I have 4 accounts - all IMAP, on different servers

I discovered a tweak that seems to avoid this issue...at least one time (Win10 > Tbird 128.7.1esr (64 bit)
I clicked on the option to have a popup confirming "OK to sync" (Settings>General>Disk Space>"Ask Every Time Before Compacting"), and that avoided the issue. I'll update this thread if I find that this work-around fails in the future.

I set the Compact option to ping me before it activated - this morning I got the message!

As soon as i selected "go" all the messages in ALL of my email accounts disappeared!!!!

See recording attached

(In reply to David Taber from comment #2)

I discovered a tweak that seems to avoid this issue...at least one time (Win10 > Tbird 128.7.1esr (64 bit)
I clicked on the option to have a popup confirming "OK to sync" (Settings>General>Disk Space>"Ask Every Time Before Compacting"), and that avoided the issue. I'll update this thread if I find that this work-around fails in the future.

David - I had also set to prompt me, but i still lost all msgs when i seleceted "Go" as per my note above, and I did not see your "workaround" until after I posted above, so it does not seem to work for me - (maybe a macOS thing, though can't see why)

Here is an intersting thing....

Ihave a Tb for each account - the vid I sent shows me looking at the other 3 accounts froim the tab of the 4th,

When I went back and opened the Inbox for the other 3 accounts from their own Tab, the msgs are there, but still none for the tab of the 4th, wheich is wqhere I was when the bug occured

Suggests that it impacts the Tab itself no matter which account I look at - the tab I am in when the compacting activates - in this case (this morning) i was in the Tab for the 4th account (Outlook) which is the one I am usually in.

I have just selected the Tab of another account and then checked the Inbox for the 4th account - messages are there!!!!!

**Deffo a TAB related bug - **

Workaround - close offending Tab and reopen

trying to get TB to request compact, if I do so from the File menu the bug does not materialise

How often does TB prompt to compact, what is the trigger?

I have lowered the threshold to a very low figure

It only seems to trigger request to run from the one and same account

Since you say things work after restart, do you get anything relevant in the Error Console when it happens (Ctrl+Shift+J)

(In reply to Magnus Melin [:mkmelin] from comment #9)

Since you say things work after restart, do you get anything relevant in the Error Console when it happens (Ctrl+Shift+J)

Magnus
See attached error report - I have had the bug since the 14.17 restart, but restroed by deleting and restarting the Tab. Not sure if you see what casued the bug in the attached report

I don't. https://hg.mozilla.org/releases/comm-esr128/file/tip/mail/base/content/about3Pane.js#l6374 doesn't look like it even could cause the error reported. (And the error may just be the result of the error situation, not what's causing it.)

Update: the Compact Request actually came up in a different Tab for a different email account - all the msgs for that account disappeared,

and - this was the same for the other email accounts using that same Tab.

If I look at the other accounts in their own Tab, the msgs are there

So the impact of the Compacting is on the Focus Tab at the time, for all of TB - but JUST that Tab

I have attached a screenshot of the error console right after the compact issues, not deleted and reinstated the affected tab at this point - there are a host more of the Cookie errors, but further up more to do with the Calendar

As the affected Tab is linked to the default account, cannot delete it, need to reboot TB

I m now using build 2 for 137.0b1

The get primarySortType error TypeError: this._sort[0] is undefined error probably explains why things go south. Why that happens...

updated to build 3 - had a prompt to compact which I ran and the bug seems to have been fixed...as all msgs still in evidence after the compacting was completed

suggest wait a day or so before looking to close this bug

nothing reported in the Daily Change logs though relating to this bug!

Please keep open, the bug just materialised again

8.40 13/03/25 - Again this morning - got same error report as Magnus notes above

08:39:29.912 Uncaught TypeError: this._sort[0] is undefined

FWIW, I've been doing the manual compacting thing for several days now, and most of the time it's doing the exactly right thing. Every once in a while the message inbox pane goes blank for a second, but then refreshes itself once the compacting cycle is complete. This is on Windows 10

(In reply to David Taber from comment #19)

FWIW, I've been doing the manual compacting thing for several days now, and most of the time it's doing the exactly right thing. Every once in a while the message inbox pane goes blank for a second, but then refreshes itself once the compacting cycle is complete. This is on Windows 10

David - what release of TB were you using?

since 137.0b2 seems to have fixed, for macOS, there is a blank for a millisec then msgs reappear

Sorry I didn't specify -- TB 128/8.0esr 64 bit

As of this morning, where compacting was still automatic, if you watch the attached video, you can see that the focus account tab, account acdp, has no messages in any folder, nor in any folder when selecting the Inbox of other email accounts - still in the same Tab

If I then select the specific account's Tab, like SPM (all 3) you can see that the messages are actually there - and if I then select the account acdp in another account's tab, messages are there as well

As noted above, the messages are lost in the account TAB that is in focus for the compacting, manual or automatic.

Attached video Screen Recording 2025-03-20 at 08.05.22.mp4

(In reply to David Taber from comment #21)

Sorry I didn't specify -- TB 128/8.0esr 64 bit

cool, tx, so seems the issue is with the beta version and from 136.0 b3 I reckon

TB now updated to 137.0b3 build 2, still have the problem with a specific Tab, when Compacting set to auto

macOS 14.7.5 (23H525)

Resolution at the moment is to reload the Tab

just updated

TB - 138.0b1
macOS - 14.7.5 (23H527)

bug still active - reset Compact to auto and with a few seconds the current account folders when blank again

(In reply to Antony from comment #26)

...
bug still active - reset Compact to auto and with a few seconds the current account folders when blank again

Would you say this is a regression?

(In reply to Wayne Mery (:wsmwk) from comment #27)

(In reply to Antony from comment #26)

...
bug still active - reset Compact to auto and with a few seconds the current account folders when blank again

Would you say this is a regression?

Yes, reckon so as i have had Compacting on auto up to the point it started to cause this bug

I see this for a single folder.

Do you ever see "The folder 'Inbox on xxxxx' could not be compacted because writing to folder failed. Verify that you have enough disk space, and that you have write privileges to the file system, then try again.

I can confirm the bug on my Laptop (Arch Linux, Thunderbird 137.0.1): the status bar displayed that Thunderbird was compacting a folder, and every folder i clicked on was empty.

A restart fixed the problem.

I am quite sure that said Laptop once also displayed the "folder ... could not be compacted because writing to folder failed" error, however i am not 100% sure that this was while i was using my windows VM - sorry.

When this happens to anyone, could you please check the error console?

I wonder if it's some JS bug that causes UI processing to abort.

I just triggred the "all folders empty" - my laptop resumed from sleep, Thunderbird started to compact folders. While the compaction started, my laptop switched from the previous connection (wifi hotspot) to ethernet, so there might have been a small window where the connection was not yet up.

Since i opened the developer console yesterday, maybe there are useful information in there? I will attach a screenshot with part of the log. I can try to keep the current Thunderbird running, since i can work without problems when i open the account inside a new tab. Inside the new tab, everything works fine - the old tab however keeps displaying empty folders even though it shows the "unread messages" count in the folderview.

Thank you for your help, have a nice day,
Michael

Based on the error log from the screenshot:

Sometimes, when calling primarySortType(), this._sort is undefined.

There are several places in mail/modules/DBViewWrapper.sys.mjs
where this._sort it set to value [] or to the results of a function.

I'm guessing it's difficult to understand why the array might be empty at a time we're trying to query [0] ?

Would it be possible return a failsafe value in primarySortType() (and other places) when it's empty?
Or does the array being empty completely break the assumptions of this code?

Hartmut, what do you think?
(I see you have touched this code several times in the past, so I'm hoping you might know.)

In other words, whenever the array is undefined, could we simply use the default order?

Patch is untested.

(In reply to Kai Engert [:KaiE:] from comment #34)

Based on the error log from the screenshot:

Sometimes, when calling primarySortType(), this._sort is undefined.

There are several places in mail/modules/DBViewWrapper.sys.mjs
where this._sort it set to value [] or to the results of a function.

I'm guessing it's difficult to understand why the array might be empty at a time we're trying to query [0] ?

The underlying cause seems to be that the message database of the folder is either missing or invalid. In the case of the error log above, accessing the database seems to fail silently here, causing the error thrown here.

This is similar to the case regarding saved searches I researched a bit in bug 1932431 comment 7 ff., while here tagged folders are involved, maybe that is of some importance for the issue.

Would it be possible return a failsafe value in primarySortType() (and other places) when it's empty?

I don't think so. The main problem seems to be that there are several code paths where a missing/invalid database isn't regenerated (by calling nsIMsgFolder::updateFolder for example) before displaying the folder.

However, the actual underlying cause is the message database becoming invalid in the first place. This has been caused in the past by running out of file handles, but for Mac OS this seems to be over 10.000, I don't know if the reporter does have that many folders. Maybe there's still something going awry in the compaction code…

By the way, this also appears to be the most common cause for reports of users having there columns and sort settings lost for some folders.

If it is happening more frequently for Mac users (or some type of users), open font files has been an issue in the past. I don't know whether it can still be a driver for open file handles. For example Bug 855751 - TB forgetting folder view and display prefs after running out of file handles on Mac. (also on linux with similar config) "The file XXXX could not be opened. ..."

(In reply to Hartmut Welpmann [:welpy-cw] from comment #39)

However, the actual underlying cause is the message database becoming invalid in the first place.

The database is likely being invalidated by the new compaction approach in Bug 1925117 - the old database is entirely blown away and the new (compacted) one installed in it's place.
There's a "compact completed" event for the main message view which tells it to rebuild using the new db, but I suspect there are other places that need something like this...

(In reply to Ben Campbell from comment #41)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #39)

However, the actual underlying cause is the message database becoming invalid in the first place.

The database is likely being invalidated by the new compaction approach in Bug 1925117 - the old database is entirely blown away and the new (compacted) one installed in it's place.

OK, I rather meant "broken" or "inaccessible"… ;)

There's a "compact completed" event for the main message view which tells it to rebuild using the new db, but I suspect there are other places that need something like this...

Yes, there is AboutToCompact" and "CompactCompleted", on which the view is destroyed and rebuilt.

To get the stack from comment 14 when selecting a folder, gDBView has to be already initialized at this place to get an error thrown here. Usually this should never be the case, since gViewWrapper is initialized only afterwards, which in turn inits gDBView.

What probably happened:

1. The folder that's currently displayed is about to be auto-compacted, so this.listener.onDestroyingView(true); gets called. Since the folder is expected to come back, gDBView is not reset here.
2. Before the compaction of the folder is completed, the user switches to another folder, so the current view wrapper has to be closed, notifying the listeners the folder is not coming back this time. However, DBViewWrapper.dbview is already null, so this does not happen here, gDBView is not null as it should be.

So not the compaction code's fault, file handles aren't to blame either…

Hartmut, thanks a lot for your analysis.

(In reply to Hartmut Welpmann [:welpy-cw] from comment #42)

So not the compaction code's fault, file handles aren't to blame either…

Who is to blame?

(In reply to Kai Engert [:KaiE:] from comment #43)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #42)

So not the compaction code's fault, file handles aren't to blame either…

Who is to blame?

The change in restoreSortIndicator() alone should be enough to fix this special case, but the stale gDBView might cause other problems too. So changing DBViewWrapper appears to be an appropriate fix, but there might be side effects. I just started a Try.

Bug 1949605 might also be a factor in why it popped up - until that landed, CompactCompleted notifications were only issued for local folders, not IMAP.

To emulate the faulty behavior from the STR in comment 42, just comment out this line.

Oh, and Bug 1949609 might be relevant: Bug 1949609 - "AboutToCompact" folder notification only sent during autocompaction and only for a single folder, even if multiple compactions are intended.

That bug hasn't been fixed, so there are probably a lot of places where AboutToCompact is never (and has never been!) called...
Not sure if that has any bearing on the comment #44 patch or not...

Hartmut: I just uploaded a patch up on Bug 1949609 which makes sure AboutToCompact is actually issued for each folder before compaction begins.
It's definitely The Right Thing To Do (tm), but I'm not sure if it'll help here, or cause more things to break.
I won't land it now unless you want it.

The discussed plan is:

• land the patch from this bug for today's build
• once today's daily build is ready, we'll send a reminder in all related bugs, and ask everyone to test with that daily build

(Afterwards, based on the results, we'll decide when to land
https://phabricator.services.mozilla.com/D247196 )

I suggest to leave this bug open after today's landing, because there are limitations regarding adding more comments, once a bug is closed, and we'll need the additional feedback.

Unfortunately this does break some tests: https://treeherder.mozilla.org/jobs?repo=try-comm-central&selectedTaskRun=SrZIawDqRby6rpstJgdoAA.0&revision=cb66c4bf64f69eb7cdfba2270464059cecfebb9a

I removed the change in DBViewWrapper, but the fix should still be effective. New Try run

The try build needs to be restarted after rust vendoring update

(In reply to Hartmut Welpmann [:welpy-cw] from comment #53)

I removed the change in DBViewWrapper, but the fix should still be effective. New Try run

I'll resubmit

(In reply to Kai Engert [:KaiE:] from comment #55)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #53)

I removed the change in DBViewWrapper, but the fix should still be effective. New Try run

I'll resubmit

Thanks, the mochitests seem to be okay.

There were some strange unit test failures in the first re-run, maybe the artifact build had used an out-of-sync binary.

I've started another one:
https://treeherder.mozilla.org/jobs?repo=try-comm-central&revision=361cf71acd8bc6fe6a38a5011fc41090dcdbee05

the new one looks good. let's land it.

Pushed by kaie@kuix.de:
https://hg.mozilla.org/comm-central/rev/cee89d02ff93
Make restoreSortIndicator() more robust. r=darktrojan

Hello everyone affected by this bug.

The latest DAILY versions from yesterday, build date 2025-05-01, contain a potential fix for this issue, but we aren't sure if the fix is sufficient.

We would greatly appreciate your help if you could please help us test that version, and give us feedback if you still see the problem with it.

When you report back, could you please indicate the version and build date you were using?
(You can find that information in the "about" dialog.)

Thank you in advance!

TB 140.0a1 (2025-05-02) (64-bit) on Linux
I now had one occurrence where automatic compacting was triggered, and the messages in the thread pane did not disappear. I wouldn't call this conclusive yet, but it is certainly encouraging.

FWIW, I'm getting the blank/disappear message list with 139 beta. In fact ALL folders then fail to display.

With 2025-05-01 daily, which has this patch, I don't get blank messages lists, and I don't get the sort error in the console. But I also didn't have automatic compact enable with earlier builds, so I can't say whether I had the issue previously. I do still get "Observed header that claims to be gloda indexed ..."

(In reply to David Taber from comment #0)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0

Steps to reproduce:

Was innocently reading an email
Compacting spontaneously began
(I've selected the option to automatically compact when savings are 100 MB)
(I doubt this makes a difference, but all mail accounts are POP3)

Actual results:

All messages in all folders no longer visible. The number of messages in messages are still displayed, but no matter where I go in the folder tree the message windows says "no messages found".
The messages are all there, but I can't see them.
Only remedy is shutdown/restart Tbird

Expected results:

During and after a folder compaction, folders should be fully functional and all messages should be shown.
This is a new bug somewhere during v128.x on Windows 10

I get this, but on IMAP. (Courier server)

(In reply to Wayne Mery (:wsmwk) from comment #62)

FWIW, I'm getting the blank/disappear message list with 139 beta. In fact ALL folders then fail to display.

With 2025-05-01 daily, which has this patch, I don't get blank messages lists, and I don't get the sort error in the console. But I also didn't have automatic compact enable with earlier builds, so I can't say whether I had the issue previously. I do still get "Observed header that claims to be gloda indexed ..."

okay, while waiting for other feedback, I decided to restart auto-compacting - issue still there, empty folders after compacting so bug 1952311 stillactive with 139.0b1 (testing bug 1963630)

(In reply to Kai Engert [:KaiE:] from comment #60)

Hello everyone affected by this bug.

The latest DAILY versions from yesterday, build date 2025-05-01, contain a potential fix for this issue, but we aren't sure if the fix is sufficient.

We would greatly appreciate your help if you could please help us test that version, and give us feedback if you still see the problem with it.

When you report back, could you please indicate the version and build date you were using?
(You can find that information in the "about" dialog.)

Thank you in advance!

is that fix in the 139.0b1 release - if yes, see above comment 64
If no, let me have the daily link and can test for you

Note: all my 4 accounts are IMAP, the issue only impacts the "active" account (selected account Tab), the other accounts are fine (though it does seem to compact them all not jsut the active as seen on the Status bar at the bottom)

• recovering is by opening the affected account in a new Tab - problem is that if its the default account a restart is needed

(In reply to Worcester12345 from comment #63)

(In reply to David Taber from comment #0)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:135.0) Gecko/20100101 Firefox/135.0

Steps to reproduce:

Was innocently reading an email
Compacting spontaneously began
(I've selected the option to automatically compact when savings are 100 MB)
(I doubt this makes a difference, but all mail accounts are POP3)

Actual results:

All messages in all folders no longer visible. The number of messages in messages are still displayed, but no matter where I go in the folder tree the message windows says "no messages found".
The messages are all there, but I can't see them.
Only remedy is shutdown/restart Tbird

Expected results:

During and after a folder compaction, folders should be fully functional and all messages should be shown.
This is a new bug somewhere during v128.x on Windows 10

I get this, but on IMAP. (Courier server)

my fix is a new Tab foir the affected account (I have 4 accounts each in its own tab - see comment 65)

my preferred layout

(In reply to Wayne Mery (:wsmwk) from comment #40)

what I end up with
only add these here is response to acooment from Wayne about the folder/font display layout maybe confusing TB after auto compacting
(In reply to Wayne Mery (:wsmwk) from comment #40)

(In reply to Wayne Mery (:wsmwk) from comment #40)

If it is happening more frequently for Mac users (or some type of users), open font files has been an issue in the past. I don't know whether it can still be a driver for open file handles. For example Bug 855751 - TB forgetting folder view and display prefs after running out of file handles on Mac. (also on linux with similar config) "The file XXXX could not be opened. ..."

Interesting thing here is that when testing the bug 1963630 I found that my message window view changed - there are some screenshots in the bug showing the change - i have msgs in standard line, whereas I end up with the tabular form

(In reply to Antony from comment #69)

(In reply to Wayne Mery (:wsmwk) from comment #40)

If it is happening more frequently for Mac users (or some type of users), open font files has been an issue in the past. I don't know whether it can still be a driver for open file handles. For example Bug 855751 - TB forgetting folder view and display prefs after running out of file handles on Mac. (also on linux with similar config) "The file XXXX could not be opened. ..."

Interesting thing here is that when testing the bug 1963630 I found that my message window view changed - there are some screenshots in the bug showing the change - i have msgs in standard line, whereas I end up with the tabular form

in reply - I now see that the Card and Table views are now showing in Appearance, which they were not 24 hrs ago!!

I prefer the Tble, but the Card view kept coming up 24 hrs ago when trying to resolve the 139 update issue in bug 1963630

Antony, sorry, but I'm not sure how your comments 64 and later are related to this issue here.
Did you try 140 daily build to see if the potential fix helps you?

Comment on attachment 9484407 [details]
Bug 1952311 - Make restoreSortIndicator() more robust. r=darktrojan

Uplift Approval Request

• Please state case for uplift consideration and ensure bug severity is set: users frequently experiencing issues with compacting folders and mail window stopping functioning completely, and we need feedback whether this fix is already sufficient, more testing is necessary
• User impact if declined: see above
• Is this code covered by automated tests?: Yes
• Has the fix been verified in Daily?: N/A
• Has the fix been verified in Beta?: No
• Needs manual test from QA?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): Geoff said it's low risk.
• String changes made/needed:

Comment on attachment 9484407 [details]
Bug 1952311 - Make restoreSortIndicator() more robust. r=darktrojan

[Triage Comment]
Approved for beta

(In reply to Kai Engert [:KaiE:] from comment #71)

Antony, sorry, but I'm not sure how your comments 64 and later are related to this issue here.
Did you try 140 daily build to see if the potential fix helps you?

Kai, I was responding to a comment from Wayne's comment in comment #40

Not tried the 140 daily yet, issues with 139 build

Thunderbird 139.0b2:
https://hg.mozilla.org/releases/comm-beta/rev/29c96c596179

(In reply to Kai Engert [:KaiE:] from comment #60)

We would greatly appreciate your help if you could please help us test that version, and give us feedback if you still see the problem with it.

When you report back, could you please indicate the version and build date you were using?
(You can find that information in the "about" dialog.)

Thank you in advance!

Hello,

i have been hitting that bug quite often. Since i installed 139.0b2 yesterday, thunderbird has survived multiple suspend-wakeup cycles without this bug triggering. I even managed to get the "Folder could not be compacted,... " dialog without having empty folderviews afterwards.

I will continue to use 139.0b2 for the next few days but am already quite confident that this bug is fixed for good.

Thank you all for your work,
Michael

Also looks fixed to me after update to 139.0b2 - thanks

Just to backup above, I am the Reporter of bug 1961074, and also posted in bug 1962262

• testing with Beta 139.0b2, Compact back on
• Still happening for me, but click away to another IMAP address, then click back, and all mail reappears.

Working as a temp fix for me.

(In reply to AlternativeKey661 from comment #78)

• Still happening for me, but click away to another IMAP address, then click back, and all mail reappears.

When this happens, could you please check what the error console says, are there any errors reported?

(In reply to Kai Engert [:KaiE:] from comment #79)

(In reply to AlternativeKey661 from comment #78)

• Still happening for me, but click away to another IMAP address, then click back, and all mail reappears.

When this happens, could you please check what the error console says, are there any errors reported?

I just had exactly this happening after a resume. I think that the autocompaction got triggered, i clicked on a folder with 1 new email indicated, and got the "No messages found". Error console shows the following:

17:45:18.940 Uncaught TypeError: gViewWrapper.dbView is null <anonymous> chrome://messenger/content/about3Pane.js:6751 isCommandEnabled chrome://messenger/content/mailCommon.js:461 getEnabledControllerForCommand chrome://messenger/content/globalOverlay.js:49 goUpdateCommand chrome://messenger/content/globalOverlay.js:79 goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109 oncommandupdate chrome://messenger/content/messenger.xhtml:1 file_init chrome://messenger/content/mailWindowOverlay.js:151 initAppMenuPopup chrome://messenger/content/mailWindowOverlay.js:1809 handleEvent chrome://messenger/content/panelUI.js:296 _openPopupPromise resource:///modules/PanelMultiView.sys.mjs:516 about3Pane.js:6751:39 <anonymous> chrome://messenger/content/about3Pane.js:6751 isCommandEnabled chrome://messenger/content/mailCommon.js:461 getEnabledControllerForCommand chrome://messenger/content/globalOverlay.js:49 goUpdateCommand chrome://messenger/content/globalOverlay.js:79 goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109 oncommandupdate chrome://messenger/content/messenger.xhtml:1 file_init chrome://messenger/content/mailWindowOverlay.js:151 initAppMenuPopup chrome://messenger/content/mailWindowOverlay.js:1809 handleEvent chrome://messenger/content/panelUI.js:296 _openPopupPromise resource:///modules/PanelMultiView.sys.mjs:516 AsyncFunctionNext self-hosted:800 17:45:18.940 An error occurred updating the cmd_downloadSelected command: [Exception... "[JavaScript Error: "gViewWrapper.dbView is null" {file: "chrome://messenger/content/about3Pane.js" line: 6751}]'[JavaScript Error: "gViewWrapper.dbView is null" {file: "chrome://messenger/content/about3Pane.js" line: 6751}]' when calling method: [nsIController::isCommandEnabled]" nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)" location: "JS frame :: chrome://messenger/content/globalOverlay.js :: getEnabledControllerForCommand :: line 49" data: yes] globalOverlay.js:81:13 goUpdateCommand chrome://messenger/content/globalOverlay.js:81 goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109 oncommandupdate chrome://messenger/content/messenger.xhtml:1 file_init chrome://messenger/content/mailWindowOverlay.js:151 initAppMenuPopup chrome://messenger/content/mailWindowOverlay.js:1809 handleEvent chrome://messenger/content/panelUI.js:296 _openPopupPromise resource:///modules/PanelMultiView.sys.mjs:516
Switching to any other folder (same imap account, same folder depth etc) resolves the issue.

@brot: Thank you.

Current line 6751 from beta corresponds to this line in c-c:
https://searchfox.org/comm-central/rev/1e8fd06088720800c214feaddf7a2225accacc25/mail/base/content/about3Pane.js#6751

It worries me that gViewWrapper.dbView can be null.

There are many references to that expression throughout the whole file.
Without knowing more details, adding a null check to all those places seems unreasonable.

I think we're learning, whatever was done recently, it opened up the possibility that gViewWrapper.dbView can be null, while related code currently assumes it will not.

I think we need a fix that ensures that gViewWrapper.dbView will not be null.
Is that possible?

Could the patch from bug 1949609 potentially prevent that gViewWrapper.dbView will be null unexpectedly?

We should differentiate two cases which may be problematic:

(a) A folder currently displayed is being compacted.
(b) A folder, that is not currently displayed, is selected while it is being compacted.

The fixes in this bug and in bug 1960349 are for case (a), while comment 80 might be case (b).

(In reply to Kai Engert [:KaiE:] from comment #81)

@brot: Thank you.

Current line 6751 from beta corresponds to this line in c-c:
https://searchfox.org/comm-central/rev/1e8fd06088720800c214feaddf7a2225accacc25/mail/base/content/about3Pane.js#6751

I think, this stack can only occur when either the message context menu or the main menu is opened. Was such an action involved, brot+mozillabt?

When I selected a folder in the process of being compacted directly after this, messages with a key higher than the one deleted before compaction couldn't be displayed anymore (empty message pane). So this really should be addressed.

It worries me that gViewWrapper.dbView can be null.

Pressing F10 after _aboutToCompactFolder and before _compactedFolder produces

Uncaught TypeError: can't access property "numSelected", gViewWrapper.dbView is null
    <anonymous> chrome://messenger/content/about3Pane.js:6751
    isCommandEnabled chrome://messenger/content/mailCommon.js:461
    getEnabledControllerForCommand chrome://messenger/content/globalOverlay.js:49
    goUpdateCommand chrome://messenger/content/globalOverlay.js:79
    goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109
    oncommandupdate chrome://messenger/content/messenger.xhtml:1
    file_init chrome://messenger/content/mailWindowOverlay.js:151
    onpopupshowing chrome://messenger/content/messenger.xhtml:1
about3Pane.js:6751:39
An error occurred updating the cmd_downloadSelected command: [Exception... "[JavaScript Error: "can't access property "numSelected", gViewWrapper.dbView is null" {file: "chrome://messenger/content/about3Pane.js" line: 6751}]'[JavaScript Error: "can't access property "numSelected", gViewWrapper.dbView is null" {file: "chrome://messenger/content/about3Pane.js" line: 6751}]' when calling method: [nsIController::isCommandEnabled]"  nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)"  location: "JS frame :: chrome://messenger/content/globalOverlay.js :: getEnabledControllerForCommand :: line 49"  data: yes] globalOverlay.js:81:13

which is similar, but not exactly the same as the output from comment 80.

(In reply to Hartmut Welpmann [:welpy-cw] from comment #83)

When I selected a folder in the process of being compacted directly after this, messages with a key higher than the one deleted before compaction couldn't be displayed anymore (empty message pane). So this really should be addressed.

I filed bug 1965686 for this.

(In reply to AlternativeKey661 from comment #78)

Just to backup above, I am the Reporter of bug 1961074, and also posted in bug 1962262

• testing with Beta 139.0b2, Compact back on
• Still happening for me, but click away to another IMAP address, then click back, and all mail reappears.

Working as a temp fix for me.

I am now experiencing the same issue as above, very odd, when in the message pane for a sub folder and Compacting starts, after requesting as I set it to prompt for permission 1st, the folder empties and only repopulates if I move to another folder and back

(In reply to Kai Engert [:KaiE:] from comment #82)

Could the patch from bug 1949609 potentially prevent that gViewWrapper.dbView will be null unexpectedly?

Probably not on it's own, but I suspect it provides the hooks that a solution would require (i.e. reliable "aboutToCompact" and notification).

(to recap: without the bug 1949609 patch, the "aboutToCompact" notification is not sent at all for non-automatic compactions. For automatic compaction, it is sent, but only for a single folder, even if multiple folders are to be compacted)

(In reply to Hartmut Welpmann [:welpy-cw] from comment #83)

(In reply to Kai Engert [:KaiE:] from comment #81)

@brot: Thank you.

Current line 6751 from beta corresponds to this line in c-c:
https://searchfox.org/comm-central/rev/1e8fd06088720800c214feaddf7a2225accacc25/mail/base/content/about3Pane.js#6751

I think, this stack can only occur when either the message context menu or the main menu is opened. Was such an action involved, brot+mozillabt?

Aside from the quick filter bar, i am quite sure that no context or main menu has been involved (been a few days and didn't know that i should look for open menus) - except that i had to open the main menu to start the developer console.

I just openend the dev console and will look if the error looks different if i am able to reproduce it.

Again, thanks for your work! Have a nice day,
Michael

(In reply to Michael from comment #87)

Aside from the quick filter bar, i am quite sure that no context or main menu has been involved (been a few days and didn't know that i should look for open menus) - except that i had to open the main menu to start the developer console.

In that case, please use the hotkey Ctrl-Shift-J to avoid producing a false positive by opening the main menu.

I just openend the dev console and will look if the error looks different if i am able to reproduce it.

Again, thanks for your work! Have a nice day,

My pleasure, same to you!

I was just able to reproduce the "empty Folder" error state, with my private inbox. However, except a whole lot of 11:53:52.229 gloda.index_msg: Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: imap://brot@WORK/INBOX/Sent sketchy key: 88 subject: Mailsubject IndexMsg.sys.mjs:1332:21 the developer console had no errors to show. The gloda errors refer to my work account, but the empty folder is my private inbox, so maybe these errors are unrelated?

Also: the Uncaught TypeError: gViewWrapper.dbView is null error is created on the click of the main menu.

Is there anything i can do to get helpful info?

(In reply to Kai Engert [:KaiE:] from comment #81)

I think we need a fix that ensures that gViewWrapper.dbView will not be null.
Is that possible?

The only way I can think of this to happen is when we are for some reason in an update batch, so that refreshing the view after the "CompactedFolder" notification fails here.

The following fix might help, but I can't really test it since I'm not able to reproduce this bug…

Okay, I think I finally found the actual cause: This line in the FolderCompactor code also closes the databases of all subfolders of the one folder that's actually to be compacted. This closes the view of any subfolder currently opened.

STR:

1. Open a folder that has subfolders and delete a message.
2. Select a subfolder of that folder.
3. Compact the parent folder.

Oof. That's pretty awful - I don't think it's obvious at all that ForceDBClosed() would close the child folder databases... but I bet there's lots of places that rely on this behaviour (it's used a lot when renaming/moving folders).
So yeah, the patch looks fine and I think we should land it.

But let's leave this bug open and apply some followup to tidy the mess:
We'll merge the new CloseDatabase() function into ForceDBClosed() and add a recursive option as a parameter.
Or better still make it non-descending, and go through the places that use ForceDBClosed() (not as many as I'd feared), and make the caller deal with child folders explicitly, rather than shortcutting everything in ForceDBClosed(). Relying on ForceDBClosed() to close child databases just muddies the logic when you're doing folder operations that affect subfolders.

Please request beta uplift quickly, if you think this would help, because we don't get sufficient testing from Daily users, we need Beta users to test.

Comment on attachment 9487164 [details]
Bug 1952311 - Do not close subfolders' databases when compacting a folder. r=BenC

Uplift Approval Request

• Please state case for uplift consideration and ensure bug severity is set: See comment 92.
• User impact if declined: When the user has opened a folder, all messages in the thread pane will be cleared when of a parent folder of it is being (auto-)compacted. Should also solve bug 1960349 in this situation.
• Is this code covered by automated tests?: No
• Has the fix been verified in Daily?: No
• Has the fix been verified in Beta?: No
• Needs manual test from QA?: Yes
• If yes, steps to reproduce: See comment 92.
• List of other uplifts needed: Bug 1949609 Actually, this is not necessarily needed, has to be rebased in that case.
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky):
• String changes made/needed: None.

Try run

I have one small change to make the patch a little bit safer, please give me some minutes.

Now the new CloseDatabase() function is exactly the same as ForceDBClosed(), just without recursively calling subfolders.

Pushed by geoff@darktrojan.net:
https://hg.mozilla.org/comm-central/rev/c28d7d49f5b4
Do not close subfolders' databases when compacting a folder. r=BenC

Comment on attachment 9487164 [details]
Bug 1952311 - Do not close subfolders' databases when compacting a folder. r=BenC

[Triage Comment]
Approved for beta

Thunderbird 139.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/313196b2c36c

(In reply to Corey Bryant from comment #102)

Thunderbird 139.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/313196b2c36c

Can you post a link when b4 can be downloaded? Happy to test

(In reply to AlternativeKey661 from comment #103)

(In reply to Corey Bryant from comment #102)

Thunderbird 139.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/313196b2c36c

Can you post a link when b4 can be downloaded? Happy to test

You can get it at https://www.thunderbird.net/en-US/thunderbird/all/ - scroll down and select it under 'Release Channel'.

(In reply to AlternativeKey661 from comment #103)

(In reply to Corey Bryant from comment #102)

Thunderbird 139.0b4:
https://hg.mozilla.org/releases/comm-beta/rev/313196b2c36c

Can you post a link when b4 can be downloaded? Happy to test

139.0b4 is available. Please let us know if any issues.

Many thanks. Added late yesterday. Early feedback -

• happening less
• however, it is still happening in "b4" multiple times

CTRL/SHIFT J errors only are

08:46:14.661 RemoteSecuritySettings: failed to download CRLite filter ServerInfoError: Server response is invalid TypeError: serverInfo.capabilities.attachments is undefined
ServerInfoError resource://services-settings/Attachments.sys.mjs:40
downloadAsBytes resource://services-settings/Attachments.sys.mjs:583
RemoteSecuritySettings.sys.mjs:631
08:46:14.662 RemoteSecuritySettings: failed to download CRLite filter ServerInfoError: Server response is invalid TypeError: serverInfo.capabilities.attachments is undefined
ServerInfoError resource://services-settings/Attachments.sys.mjs:40
downloadAsBytes resource://services-settings/Attachments.sys.mjs:583
RemoteSecuritySettings.sys.mjs:631
08:46:14.662 RemoteSecuritySettings: failed to download CRLite filter ServerInfoError: Server response is invalid TypeError: serverInfo.capabilities.attachments is undefined
ServerInfoError resource://services-settings/Attachments.sys.mjs:40
downloadAsBytes resource://services-settings/Attachments.sys.mjs:583

09:21:14.060
NS_ERROR_FAILURE: Ignore PDF.js for this download. PdfStreamConverter.sys.mjs:1074
09:21:28.815
sendRemoveListener on closed conduit languagetool-mailextension@languagetool.org.61 3 ConduitsChild.sys.mjs:122:13

11:09:43.184 sendRemoveListener on closed conduit languagetool-mailextension@languagetool.org.73 3 ConduitsChild.sys.mjs:122:13

11:14:38.910
WebExtension context not found! ExtensionParent.sys.mjs:1364:13
11:14:38.911
sendRemoveListener on closed conduit languagetool-mailextension@languagetool.org.85 3 ConduitsChild.sys.mjs:122:13

After 2nd time
12:46:49.022 NotFoundError: No such JSProcessActor 'BrowserToolboxDevToolsProcess'
12:51:14.707 [issue 351] TB115 - to do: QuickFolders_MySelectFolder(): toggleRow for parent folder: [object HTMLLIElement] quickfolders-util.js:1322:13
12:51:39.164 NotFoundError: No such JSProcessActor 'BrowserToolboxDevToolsProcess'

​

Happened again (but only mentions Quickfolders)
14:23:49.231 [issue 351] TB115 - to do: QuickFolders_MySelectFolder(): toggleRow for parent folder: [object HTMLLIElement] quickfolders-util.js:1322:13

And again
15:47:26.272 sendRemoveListener on closed conduit languagetool-mailextension@languagetool.org.71 3 ConduitsChild.sys.mjs:122:13

All of these logging seems unrelated. FWIW, it is expected for the message list to be cleared while the folder is being compacted, which can take some time, depending on its size. So, the message list doesn't reload itself after some time?

(In reply to Hartmut Welpmann [:welpy-cw] from comment #109)

All of these logging seems unrelated. FWIW, it is expected for the message list to be cleared while the folder is being compacted, which can take some time, depending on its size. So, the message list doesn't reload itself after some time?

Just happened again this morning. No, staying on the "inbox" affected it does not rebuild and have messages then appear. The only solution is to click away to another folder or inbox, then back. Perhaps my issues relate more to the bug 1961074 I posted?

This was the errors (last line only was blue)
08:33:47.095 {LISTENERS.TABMAIL} select category __ALL quickfolders-tablistener.js:37:21
08:33:47.233 WebExtensions: qf-3pane.js - onLoad() qf-3pane.js:59
08:33:48.245 WebExtensions: QuickFolders: injecting current folder qf-3pane.js:72
08:34:18.159 [issue 351] TB115 - to do: QuickFolders_MySelectFolder(): toggleRow for parent folder: [object HTMLLIElement] quickfolders-util.js:1322:13

Could you run Beta 4 for a while in troubleshoot mode (without any add-ons) and see if that helps?

(In reply to Hartmut Welpmann [:welpy-cw] from comment #111)

Could you run Beta 4 for a while in troubleshoot mode (without any add-ons) and see if that helps?

Tested in Troubleshooting. Had it happen (first time so far and been testing for 2 hours)
10:53:44.301 services.settings: Remote Settings startup changesets bundle could not be extracted (TypeError: serverInfo.capabilities.attachments is undefined) remote-settings.sys.mjs:227
10:53:44.302 services.settings: UnknownCollectionError: Unknown Collection "thunderbird/url-classifier-exceptions"
UnknownCollectionError resource://services-settings/RemoteSettingsClient.sys.mjs:189
sync resource://services-settings/RemoteSettingsClient.sys.mjs:635
RemoteSettingsClient.sys.mjs:555

and happened again
13:13:50.329 NotFoundError: No such JSProcessActor 'BrowserToolboxDevToolsProcess'

took back to normal for a bit to do stings then got this when reset to TS mode

15:00:40.571
UnknownCollectionError: Unknown Collection "thunderbird/moz-essential-domain-fallbacks"
UnknownCollectionError resource://services-settings/RemoteSettingsClient.sys.mjs:189
sync resource://services-settings/RemoteSettingsClient.sys.mjs:635
_beforeUIStartup resource:///modules/MailGlue.sys.mjs:610
observe resource:///modules/MailGlue.sys.mjs:473
EssentialDomainsRemoteSettings.sys.mjs:110:15
15:00:40.572
UnknownCollectionError: Unknown Collection "thunderbird/url-parser-default-unknown-schemes-interventions"
UnknownCollectionError resource://services-settings/RemoteSettingsClient.sys.mjs:189
sync resource://services-settings/RemoteSettingsClient.sys.mjs:635
_beforeUIStartup resource:///modules/MailGlue.sys.mjs:610
observe resource:///modules/MailGlue.sys.mjs:473
SimpleURIUnknownSchemesRemoteObserver.sys.mjs:114:15
15:00:59.122 services.settings: Remote Settings startup changesets bundle could not be extracted (TypeError: serverInfo.capabilities.attachments is undefined) remote-settings.sys.mjs:227
15:00:59.124 services.settings: UnknownCollectionError: Unknown Collection "thunderbird/url-classifier-exceptions"
UnknownCollectionError resource://services-settings/RemoteSettingsClient.sys.mjs:189
sync resource://services-settings/RemoteSettingsClient.sys.mjs:635

It will be a weekend for me soon, so it will be Monday before post again (unless these are not helpful)

I don't think that further reporting of unrelated console errors is helpful, especially not at comment 114 of a bug.

The remote settings errors (comment 114, comment 112, 106) were fixed in bug 1954911 and bug 1927066). Errors caused by add-ons (Quickfolders, comment 110 or Language Tool, comment 108) are not relevant here.

The fix in comment 59 fixed the "gViewWrapper.dbView is null", last mentioned in comment 83. The Gloda errors from the screenshot in comment 33 were fixed in bug 1963490.

Finally, TypeError: this._sort[0] is being looked at in bug 1932431. In my experience, when this error is received, all bets are off and switching folders doesn't display their content any more.

So please use a version of TB where the aforementioned bugs are all fixed, and if you can still reproduce "messages in all folders no longer visible", file another bug.

I noticed that the fix from comment 100 which changes ForceDBClosed() addresses a problem introduced here in bug in TB 137. So that fix doesn't address any issue encountered in TB 128.

The other checks above here do consider null gViewWrapper.dbView which is something we saw in this bug.

Pushed by alessandro@thunderbird.net:
https://hg.mozilla.org/comm-central/rev/f5319569440f
Protect against null gViewWrapper.dbView also for "cmd_downloadSelected". r=welpy-cw

(In reply to Francesco from comment #115)

So please use a version of TB where the aforementioned bugs are all fixed, and if you can still reproduce "messages in all folders no longer visible", file another bug.

I am using 139.0b4 and as requested by Hermut in Comment 112 in Troubleshooting Mode as well (as some had not been in this)
I did post a bug about the issues and was asked to post here along with others using the beta which i have done for the last few weeks.
I will drop from the thread as it seems the requested feedback (I have no idea to tell what ones you need and what is of no use) nor anyway to send outside this forum. Good luck with the fix as for me this has had huge impact hence the desire to try help (my first time with the systems requirements so apologies to all if I have offended).

TB139.0, Release update channel on Linux.
When automatic compacting was triggered, messages in the current selected folder were no longer visible. Switching folders did restore the message list in the affected folder though.

From the error console (slightly redacted):
gloda.index_msg: Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: imap://<redacted>@imap.mail.yahoo.com/Read sketchy key: 176311 subject: <redacted> IndexMsg.sys.mjs:1332:21
gloda.index_msg: Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: imap://<redacted>@imap.mail.yahoo.com/Read sketchy key: 176310 subject: <redacted> IndexMsg.sys.mjs:1332:21
gloda.index_msg: Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: imap://<redacted>@imap.mail.yahoo.com/Read sketchy key: 176309 subject: <redacted> IndexMsg.sys.mjs:1332:21
gloda.index_msg: Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: imap://<redacted>@imap.mail.yahoo.com/Read sketchy key: 176316 subject: <redacted> IndexMsg.sys.mjs:1332:21

Uncaught TypeError: gViewWrapper.dbView is null about3Pane.js:6755:39
<anonymous> chrome://messenger/content/about3Pane.js:6755
isCommandEnabled chrome://messenger/content/mailCommon.js:461
getEnabledControllerForCommand chrome://messenger/content/globalOverlay.js:49
goUpdateCommand chrome://messenger/content/globalOverlay.js:79
goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109
oncommandupdate chrome://messenger/content/messenger.xhtml:1
file_init chrome://messenger/content/mailWindowOverlay.js:151
initAppMenuPopup chrome://messenger/content/mailWindowOverlay.js:1809
handleEvent chrome://messenger/content/panelUI.js:296
_openPopupPromise resource:///modules/PanelMultiView.sys.mjs:516

<anonymous> chrome://messenger/content/about3Pane.js:6755
isCommandEnabled chrome://messenger/content/mailCommon.js:461
getEnabledControllerForCommand chrome://messenger/content/globalOverlay.js:49
goUpdateCommand chrome://messenger/content/globalOverlay.js:79
goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109
oncommandupdate chrome://messenger/content/messenger.xhtml:1
file_init chrome://messenger/content/mailWindowOverlay.js:151
initAppMenuPopup chrome://messenger/content/mailWindowOverlay.js:1809
handleEvent chrome://messenger/content/panelUI.js:296
_openPopupPromise resource:///modules/PanelMultiView.sys.mjs:516
InterpretGeneratorResume self-hosted:1425
AsyncFunctionNext self-hosted:800

An error occurred updating the cmd_downloadSelected command:
[Exception... "[JavaScript Error: "gViewWrapper.dbView is null" {file: "chrome://messenger/content/about3Pane.js" line: globalOverlay.js:81:13
6755}]'[JavaScript Error: "gViewWrapper.dbView is null" {file: "chrome://messenger/content/about3Pane.js" line: 6755}]' when calling method: [nsIController::isCommandEnabled]" nsresult: "0x80570021
(NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)" location: "JS frame :: chrome://messenger/content/globalOverlay.js :: getEnabledControllerForCommand :: line 49"
data: yes]
goUpdateCommand chrome://messenger/content/globalOverlay.js:81
goUpdateMailMenuItems chrome://messenger/content/mailWindowOverlay.js:109
oncommandupdate chrome://messenger/content/messenger.xhtml:1
file_init chrome://messenger/content/mailWindowOverlay.js:151
initAppMenuPopup chrome://messenger/content/mailWindowOverlay.js:1809
handleEvent chrome://messenger/content/panelUI.js:296
_openPopupPromise resource:///modules/PanelMultiView.sys.mjs:516

Reporter, davofanmail, seems to be gone.

I have waited a bit to see if i am really reproducing the problem, but i am able to reproduce the bug on 139.0b4 (64-Bit).
It seems to trigger whenever thunderbird is otherwise busy (for example running compact). I am then shown the usual empty folder, however instead of having to open the inbox in an new tab, i can just click on any other folder and back onto the currently empty one and it will show the mails.
Is there anything i can to to help to debug this further?
Thanks again for everyones help, have a nice day,
Michael